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1.0  PROGRAM  SUMMARY  AND  INTRODUCTION  -  CPOM 


1.1  Introduction 

Northrop  Grumman  has  been  pleased  to  carry  out  the  Commercial  Parts  Obsolescence 
Management  (CPOM)  program  under  Cooperative  Agreement  F336 15-99-2-5500.  Under  this 
agreement,  AFRL  Manufacturing  Technology  Division  (ManTech)  provided  approximately  75% 
of  the  program  funding,  and  Northrop  Grumman  provided  25%.  Over  the  course  of  the  program, 
we  exceeded  the  required  cost  share  by  27%.  The  program  has  been  completed  successfully 
within  the  funding  provided. 

CPOM  began  in  June  1999,  and  was  scheduled  to  run  for  58  months,  with  an  April  2004 
completion.  The  program  has  been  kept  on  schedule,  with  no  delays  or  extensions,  and  is  now 
being  completed  a  couple  of  months  early  with  the  delivery  of  this  Final  Report. 

The  program  was  originally  compiled  from  several  proposals  in  different  areas  that  were 
merged  together.  With  this  background,  it  has  been  an  extremely  diverse  effort  whose  broad 
scope  has  provided  many  benefits  to  Northrop  Grumman  and  our  customers.  The  first  phase  of 
the  program,  called  ACME  for  Application  of  Commercially  Manufactured  Electronics, 
developed  a  wide  range  of  processes,  tools,  and  analyses.  The  second  phase  of  the  program,  the 
Pilot,  successfully  carried  out  its  charter  of  documenting  the  benefits  of  the  tools  and  processes 
in  spite  of  a  major  redirection. 

Since  1999,  we  have  submitted  17  Quarterly  Reports  that  have  provided  detailed 
information  and  documentation  of  results.  These  reports  are  a  treasure  trove  of  information  on 
reliability,  use  and  testing  of  COTS  parts,  process  improvements,  evaluation  of  groundbreaking 
tools,  and  analysis  of  chips  and  packages.  In  addition,  we  have  participated  in  many  Electronic 
Parts  Obsolescence  Initiative  (EPOI)  meetings  to  further  share  information  with  other 
contractors  and  members  of  the  obsolescence  community.  Several  presentations  have  also  been 
made  to  AFRL  personnel  to  highlight  program  successes  and  demonstrate  the  tools  we  have 
developed. 

We  now  present  this  Final  Report  to  succinctly  summarize  the  benefits  and  successes  that 
CPOM  has  attained.  Comments  and  questions  are  invited  to  the  contacts  below: 


Erik  Rennenkampf 
CPOM  Program  Manager 
410-765-2912 

erik.rennenkampf@ngc.com 


Rich  Rhyne 

CPOM  Engineering  Manager 

410-765-2210 

richard.rhyne@ngc.com 
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1.2  Executive  Summary 

CPOM  addressed  existing  COTS  obsolescence  tools  and  process  issues  under  the 
following  headings. 

•  General  Plastic  Encapsulated  Microcircuit  (PEM)  Issues  -Qualification  of 
commercial  components. 

•  CPOM  Packaging  Issues  -  Application  and  testing  of  COTS  parts. 

•  Reliability  Prediction  Comparisons  -  Evaluation  and  implementation  of  reliability 
models. 

•  Recommendations  &  Design  Guidelines  -  Design  rules  and  design  capture 

•  Design  Of  The  Environment  -  COTS  part  bonding  and  thermal  issues 

•  Tool  Development  Assistance  -  Interaction  with  the  tool  developers  of  RADSS, 
DVTG,  i2,  MOCA  and  VP  Technologies  models. 

•  Titan  Corporation  Subcontract  -  Develop,  integrate  and  test  Poet  software  and  Rosetta 
models.  Integrate  tools  and  Rosetta  models  using  the  Poet  software  suite.  A 
workflow  task  was  added  in  2003  to  automate  Bill  of  Material  analysis  and  design 
review  checklists. 

•  Baseline  Legacy  System/Metrics  -  Establish  Quantifiable  Metrics,  Collect  baseline 
data. 

•  Improved  Policies/Practices/  Procedures  -  Process  improvements,  Lean  Corporate 
Processes/Practices. 

•  MOCA  Evaluation  -  Evaluate  MOCA  on  NGC  test  case  and  integrate  MOCA  with 
Poet. 

•  Georgia  Tech  Subcontract  -  Develop  and  verify  Physics-based  virtual  reliability 
models. 

Through  the  implementation  of  improved  tools  and  processes,  CPOM  was  able  to 
demonstrate  more  than  $22M  total  cost  avoidance  across  multiple  platforms  on  the  pilot 
program. 
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Table  1-1  CPOM  Cost  Savings  Demonstrated  Across  Northrop  Grumman  Electronic 

Systems 


Tool 

Previous  Cost 

Approach  with  CPOM 

Savings 

i2 

Revision  Notices 

Military  parts  required 

Manual  New  Part  Request 
and  Review 

Evaluate  part  number  and 

DMS  issues  at  part  selection 

COTS  enabled 

Electronic  NPR  workflow, 
integrated  to  design  tools 

Revision  cost 
avoidance  $>1.8M 
annually 

$>3M  annually 

See  CDT  metric 

CDT 

$9,400  per  new  part 

from  Coopers  and 

Lybrand 

Use  CDTs  to  standardize  parts 

S-6.8M  annually  in 
avoidance  of  new  part 
introductions 

Reliability  Prediction 

Program  specific 

Reduced  search  time, 
improved  prediction,  customer 
acceptance  of  methodology 

-$300,000  annually. 
Enables  additional 

55%  of  IC  technology 

CPT/Design  Reuse 

Program  IPT 

CPT  product  reviews  and 
standardization 

$>11M  in  ’03.  23% 
design  cycle  time 
reduction  for  some 
products 

RADSS 

Minor  redesign  $22.4K  to 
$11  IK 

Major  redesign 

$200K  to  $770K 

from  EIA  bulletin  GEB 1 

Demonstration  tool.  Avoid 
redesigns 

Pilot  case  shows 
potential  for  ~$500K 

DVTG  (VectorGen) 

VHDL  capture 

Demonstration  tool  only 

Potential  savings  in 
reduction  of  redesign 
cost 

VP  Technologies 

Typically  not  modeled 

Demonstration  tool  only 

Potential  savings  at  AF 
Program  Office  level 
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1.3  Vision  Statement 

The  primary  objective  of  CPOM  was  to  document  the  cost  avoidance  resulting  from  the 
implementation  of  obsolescence  management  tools,  reliability  tools,  corporate  initiative  tools 
and  lean  corporate  practices. 

1.4  Goals  and  Objectives 

In  order  to  achieve  the  desired  cost  reductions,  CPOM  was  required  to  demonstrate 
reliable  use  of  Commercial  Off-the-Shelf  (COTS)  parts  in  military  systems,  implement  USAF 
ManTech  funded  tools  and  processes  and  Integrate  tools  and  processes  in  the  Northrop 
Grumman  Electronic  Systems  Integrated  Design  Environment  (IDE) 

2.0  TECHNICAL  TASKS  AND  RESULTS 

2.1  General  PEM  Issues,  Design  Capture,  Packaging  Issues,  Design  of  the 
Environment 

CPOM  established  component  test  methodologies  and  qualification  thresholds  utilizing 
industry  models  and  physics  of  failure  analysis.  When  the  program  was  initiated,  we  could  find 
no  industry  experts  who  fully  understood  the  various  models.  As  the  program  progressed  the 
models  were  rederived  for  various  military  application  environments.  The  end  results  are  very 
close  to  the  models  that  are  now  defined  in  AEC  Q100,  STACK  001  and  SSB-1.  Along  the  way, 
a  great  deal  was  learned  about  how  to  apply  the  models  in  component  selection  and  we  gained 
the  detailed  knowledge  to  explain  the  processes  to  our  customers,  engineers  and  suppliers. 

Northrop  Grumman  completed  the  baseline  documentation  of  end  of  life  failure 
mechanisms  unique  to  PEM  and  incorporated  the  corrosion  failure  mechanism  data  and  thermal 
stress  failure  mechanism  data  into  our  component  qualification  threshold  requirements.  Non¬ 
unique  failure  mechanisms  are  addressed  in  the  same  manner  as  has  been  done  with  hermetically 
sealed  devices. 

2.2  Packaging  Issues 

2.2.1  Chip-to-Board  CTE  Mismatch  Testing 

Chip-to-Board  CTE  Mismatch  Testing  was  designed  to  fine  tune  design  rules  and 
establish  thresholds  for  chips  size  versus  board  material  to  avoid  thermomechanical  fatigue 
failures.  The  primary  focus  of  this  testing  was  on  chip  devices  and  leadless  chip  carriers 
mounted  on  various  substrates.  Design  rules  were  developed  and  incorporated  into  design 
reviews  and  new  part  selection  processes.  Since  single  layer,  alumina  based  resistor  chips 
represented  the  biggest  potential  problem  area,  that  analysis  was  automated  in  the  Bill  of 
Material  analysis  tool  by  Titan  Corporation. 
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2.2.2  40-mil  BGA  Testing 

40-mil  BGA  Testing  -  Testing  of  various  BGA  sizes,  pitches  and  constructions  was 
performed  under  CRAD  and  IRAD  available  from  other  contracts  and  from  Northrop  Grumman 
internal  funding.  This  information  was  incorporated  into  design  rules  and  design  review 
procedures  developed  under  CPOM.  A  BGA  steering  committee  was  established  to  review  new 
BGA  packages  selected  for  introduction  into  Northrop  Grumman  designs. 

2.2.3  COTS  Test  Vehicle 

The  goal  of  this  exercise  was  to  evaluate  a  variety  of  COTS  BGA  formats  for  assembly 
and  reliability  and  to  determine  the  reliability  impact  of  various  underfills  and  mounting 
approaches.  Candidate  BGA,  CSP  COTS  components  and  underfills  were  selected. 
Environmental  test  criteria  were  defined  and  a  statistical  test  plan  was  developed  that  included 
thermal  cycling  and  vibration  testing.  The  test  results  showed  that  underfill  helps,  but  is  not 
required  for  near  CTE-matched  COTS  parts  to  meet  reliability  requirements.  Northrop 
Grumman  evaluates  the  thermal  compatibility  each  assembly  and  avoids  underfill  where  possible 
in  order  to  ease  manufacturability  and  decrease  cost.  Underfill  improved  reliability  for  all  cases 
except  ceramic  column  grid  arrays.  We  noted  that  “cavity”-down  perimeter  BGAs  performed 
better  than  “cavity”-up  full-area  BGAs  and  that  reliability  improvements  for  ceramic  BGA 
assemblies  were  not  sufficient  to  meet  program  reliability  requirements. 

2.2.4  Alternate  ASIC  &  MMIC  Packaging  Evaluation. 

The  goal  of  this  research  was  to  establish  selection  criteria  for  best  format  for  COTS 
ASIC  and  MMIC  packaging  given  alternate  available  configurations,  to  define  preferred 
packaging  format  for  near-term  applications  and  to  establish  a  roadmap  for  future  format 
preferences.  Internal  and  external  surveys  of  parts  preferences  were  performed  for  design 
engineering  and  manufacturing.  Literature  searches  and  reviews  were  conducted  for  preferences 
with  respect  to  performance,  reliability,  and  industry  packaging  trends.  Near-term  and  long-term 
packaging  format  preferences  were  defined  and  design  guidelines  were  modified  to  reflect 
findings. 

While  the  flip  chip  format  offers  superior  performance  and  packaging  densities,  current 
infrastructure  does  not  support  full  implementation.  Near-term  parts  selection  is  restricted  to 
BGA  and  CSP  formats  having  0.75  mm  pitch  or  greater,  with  smallest  pitch  parts  preferred. 

High  density  interconnect  (HDI)  printed  wire  board  technology  is  needed  to  use  finer  than  0.75 
mm  pitch  components,  but  reliability  of  these  is  still  being  determined. 

2.2.5  COTS  RF  &  Digital  Connectors 

The  goal  of  this  design  effort  was  to  evaluate  adaptation  of  COTS  RF  and  Digital 
connectors  for  military  environments  and  to  reduce  the  cost  of  RF  connectors. 

The  team  worked  with  Amphenol,  designing  custom  inserts  for  their  standard  product 
offering  to  accommodate  several  military  program  requirements  for  backplane  connectors. 

Molds  for  these  inserts  were  designed  and  procured.  Upon  receipt,  these  were  tested  for 
mechanical  fit  and  electrical  performance.  The  new  connectors  were  subjected  to  environmental 
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and  reliability  testing  for  insertion  into  future  products.  These  connectors  have  become  the 
standard  connectors  for  module  interfaces. 

An  additional  team  activity  was  to  work  with  COTS  RF  connector  suppliers  Gore  and 
Gilbert  Engineering  by  designing  and  prototyping  new,  smaller  high  performance  RF  connectors. 
The  new  connectors  are  60%  the  size  of  Gilbert  GPPO  connectors.  These  are  planned  for 
commercialization  by  Gore  under  their  Mini-SMP-100  product  line. 

Sample  connectors  were  delivered  and  parts  were  successfully  incorporated  into  the 
design  of  one  of  our  common  radar  modules. 

Concurrently,  testing  was  conducted  on  these  parts  by  Gore.  As  expected,  performance 
of  these  parts  exceeded  current  GPO/SMP  and  GPPO/Mini-SMP  specifications  with  no 
substantial  increase  in  cost,  while  significantly  reducing  the  size. 

2.2.6  Design  of  the  Environment 

Design  of  the  Environment.  Packaging  evaluation  was  undertaken  as  a  means  of 
understanding  the  impact  on  manufacturing  processes  driven  by  the  use  of  COTs  components. 
Many  misunderstandings  existed  with  respect  to  COTs  packaging  consistency.  Dimensions  and 
stand-off  height  were  measured  and  determined  to  be  very  consistent  across  manufacturers  for 
components  in  the  same  family,  same  part  number  and  same  package  types.  This  put  to  bed  a 
popular  excuse  for  not  using  COTs  and  enabled  manufacturing  processes  to  be  standardized  and 
documented.  It  was  determined  that  the  majority  of  inconsistencies  found  on  the  PEMs’  stand¬ 
off  height  were  due  to  handling  issues  and  could  be  addressed  using  basic  material  handling 
controls.  Northrop  Grumman  documents  packaging  for  automated  handling  in  the  product  data 
management  system  and  electronically  flows  it  down  to  suppliers  on  the  purchase  order.  This 
provides  an  additional  checkpoint  for  design  reviews  as  packaging  information  is  readily 
available  to  manufacturing  well  before  any  material  is  placed  on  order  or  delivered  to  the  factory 
floor.  Upon  receipt  packaging  is  verified  and  stocked  in  as  received  condition  thus  eliminating 
multiple  handling  and  repackaging  steps. 

Die  orientation  within  integrated  circuit  packages  was  evaluated  for  impact  on  thermal 
dissipation  characteristics.  The  results  were  used  to  fine  tune  thermal  analysis  and  thermal 
management  techniques. 

The  use  of  conductive  epoxies  was  evaluated,  selecting  epoxies  for  thermal  conductivity, 
the  lift  the  adhesive  gives  to  a  PEM,  reworkability,  and  the  shelf  and  pot  life  of  the  mixed 
compounds. 

Bonding  is  often  necessary  for  thermal  dissipation  or  vibration  damping;  however,  it 
requires  additional  manufacturing  steps  that  drive  system  cost  up.  Because  of  the  additional 
cost,  bonding  should  be  employed  only  when  analysis  indicates  it’s  a  necessity.  Board  designs 
should  contain  enhanced  thermal  paths  to  reduce  or  eliminate  the  amount  of  epoxy/adhesive. 
When  applying  adhesive,  general  adhesive,  generally,  a  matrix  of  dots  gives  better  results  than 
does  either  a  series  of  lines  or  a  solid  area. 

While  not  yet  in  wide  spread  usage,  the  concept  of  a  spray  cooled  board  proved  to  be  an 
effective  method  depending  on  board  population  density.  Spray  cooling  allows  the  use  of  low 
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cost  air  cooled  COTs  boards.  It  also  provides  an  increase  in  heat  transfer  density  as  compared  to 
conduction  cooling  or  standard  air  cooling.  The  evaporation  of  the  Flourinert  spray  provides  a 
large  heat  capacity.  This  improved  thermal  capability  provides  lower  device  junction 
temperatures  which  can  improve  the  device  reliability.  For  thermally  limited  systems,  the  spray 
cooling  can  provide  growth  capability  for  additional  processor  boards. 

Helium  cooling  was  also  evaluated  and  while  theoretically  valid,  it  proved  to  be  difficult 
to  implement.  The  circuit  board  is  enclosed  with  a  sealed  cover  on  the  module  and  the  space 
filled  with  helium  to  enhance  heat  transfer.  The  thermal  conductivity  of  helium  is  more  than  five 
times  the  conductivity  of  air,  at  temperatures  typical  for  electronic  components.  This  provides  a 
better  conduction  path  from  the  component  to  the  printed  circuit  board  and  cover  which  are 
thermally  connected  to  the  heat  sink,  located  at  the  edge  of  the  printed  circuit  board.  The 
concept  of  helium  cooling  is  sound;  however,  it  produces  two  major  drawbacks.  Helium  cooled 
boards  are  very  expensive  and  it  is  extremely  difficult  to  achieve  a  seal  to  retain  the  helium. 

2.3  Reliability  Prediction  Process  Flow 

The  goal  of  CPOM  reliability  prediction  analysis  was  to  improve  the  existing  reliability 
prediction  methodology,  specifically  addressing  commercial  parts  in  military  applications.  The 
approach  was  to  use  field  data  to  identify  improvements  in  prediction  modeling  assumptions  and 
to  develop  a  standardized  prediction  process  based  on  modeling  enhancements,  yielding  credible 
MTBF  results. 

An  independent  assessment  of  field  data  collected  on  plastic  parts  used  in  a  military, 
airborne  environment  was  performed.  Northrop  Grumman  then  generated  predictions  using 
traditional  methods  (MIL-HDBK-217FN2)  and  alternative  models  (RAC  PRISM®),  comparing 
findings  to  field  data  from  the  MODAR  program. 

This  exercise  provided  justification  to  modify  MIL-HDBK-217FN2  factors  for  better 
correlation  to  field  data  and  more  accurate  inputs  to  obsolescence  management  decision  tools. 

The  RAC  PRISM®  tool  was  evaluated  against  new  designs  to  serve  as  baseline  for 
evaluating  future  prediction  methods. 

An  improved  prediction  process  was  established  utilizing  a  standardized  methodology 
and  sharing  of  component  data  across  programs. 

Results  showed  that  MIL-HDBK-217FN2  predictions  based  on  kq  factors  of  2  or  3  were 
within  10%  of  the  demonstrated  field  reliability  for  the  CCAs  in  the  study.  This  observation 
provided  the  basis  to  use  "better  than"  standard  jiq  factors  when  predicting  reliability  for  plastic 
components.  The  modified  factors  are  contingent  upon  a  disciplined  parts  selection  process  and 
proper  attention  to  component  derating  practices. 

PRISM®  predictions  correlated  better  to  the  field  data  when  the  duty  cycle  and  cycling 
rate  default  values  were  modified  to  more  closely  represent  the  field  operating  conditions.  On 
average,  the  predictions  were  20%  to  40%  closer  to  the  field  data  compared  to  the  initial 
predictions  based  on  the  PRISM®  environmental  default  values. 
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RACRates®  models  embedded  in  the  PRISM®  tool  should  be  used  to  the  maximum 
extent  possible.  Our  experience  revealed  the  failure  rates  from  the  RAC  database  skewed  the 
overall  prediction  for  one  of  the  CCAs  in  the  study.  This  observation  confirmed  the  need  for 
additional  RACRates®  models  if  the  PRISM®  tool  is  to  become  a  viable  method  for  predicting 
the  reliability  of  complex  systems. 

PRISM®  predictions  produced  overly  optimistic  results  when  the  System  Level 
Multiplier  was  incorporated  in  the  analysis.  Our  recommendation  is  to  continue  generating 
predictions  with  and  without  the  System  Level  Multiplier,  yet  refrain  from  formally  using  it  until 
field  data  is  available  to  substantiate  the  factors.  Specific  knowledge  of  the  program's  design, 
manufacturing,  quality  and  management  practices  is  required  for  an  accurate  assessment. 


Figure  2-1  A  typical  Result  for  one  of  the  Two  CCAs  in  the  Study 

The  range  of  data  points  shows  how  the  predictions  varied  according  to  assumptions  and 

methods. 

The  MIL-HDBK-217FN2  method  which  best  correlated  to  the  field  data  was  based  on  a 
quality  factor  of  2.  The  PRISM®  prediction  which  came  closest  to  the  field  data  was  obtained 
after  modifying  the  default  values  and  excluding  the  effects  of  the  System  Level  Multiplier 
(process  grade  factors). 

2.4  Improved  Policies/Practices/Procedures 
2.4.1  CSM  Implementation  (ITA) 

Parametric  component  search  and  selection,  a  common  Standard  Parts  List  and  a 
common  new  part  qualification  procedure  were  implemented  using  the  i2  Component  and 
Supplier  Management  tool.  This  allows  for  electronic  collaboration  at  Electronic  Systems  sites 
across  the  country. 
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2.4.2  Integrate  CSM  to  Design  Tools  (IT A) 

The  i2  New  Part  Request  workflow  was  electronically  integrated  to  the  ECAD  model 
and  board  layout  library  process.  A  request  for  a  new  CAD  model  automatically  triggers  a  New 
Part  Request.  The  part  number  and  basic  part  metadata  are  validated  prior  to  the  generation  of 
the  CAD  model.  Full  part  qualification  proceeds  in  parallel  with  the  model  creation. 

2.4.3  E  -  Procurement  Model  (ITA) 

Procurement  cycle  time  for  standard  parts  was  reduced  and  designers  were  encouraged  to 
use  standard  parts  by  automating  the  electronic  procurement  of  these  devices.  SPL  parts  are 
placed  on  corporate  contract  by  procurement  with  pricing  having  been  prenegotiated.  These 
parts  also  appear  in  electronic  catalogs  that  allow  designers  to  directly  “order”  the  materials  they 
need  for  developing  designs.  While  not  used  as  a  production  procurement  system,  this  approach 
has  helped  to  get  engineers  working  material  and  increase  standard  part  usage. 

2.4.4  DMS  Work  Flow 

i2  Life  Cycle  Management  System  (LMS)  data  is  used  many  times  during  a  program’s 
development.  The  LMS  data  is  evaluated  as  a  part  of  the  New  Part  Request.  It  is  evaluated 
again  during  the  design  review  process.  A  Bill  of  Material  can  be  analyzed  by  exporting  the 
product  structure  from  the  product  data  management  system  and  electronically  comparing  it  to 
the  i2  life  cycle  data.  Automation  of  the  process  has  been  demonstrated  using  Titan’s  Poet 
software  suite. 

2.4.5  Electronic  Design  Model  Library  (ITA) 

As  a  means  of  reducing  design  cycle  time,  CPOM  sponsored  the  creation  of  ECAD 
models  for  items  added  to  the  Standard  Parts  Lists  by  the  Commodity  Design  Teams.  This  made 
the  devices  readily  available  to  engineers  without  the  need  for  an  ECAD  model  request. 

2.4.6  RF  SPL  and  Qual  (1/2  ITA) 

The  RF  Commodity  Design  Team  expanded  the  classic  definition  of  PEM  to  extend  the 
concepts  developed  for  qualification  of  digital  ICs  to  the  RF  world.  Qualification  requirement 
thresholds  were  developed  and  published.  The  RF  Standard  Parts  List  was  released  into  the  i2 
Explore  software  and  is  available  sector  wide  to  assist  designers  in  selection  of  qualified 
components  for  use  in  our  applications.  The  Commodity  Design  Team  is  using  the  i2  Explore 
NPR  routing  and  approval  tool  as  noted  above.  The  RF  Commodity  Design  Team  performed 
many  technology  reviews  with  preferred  suppliers,  soliciting  their  input  into  the  RF  SPL. 

2.4.7  Design/Hardware  Reuse 

Reuse  of  design  in  the  form  of  Modular  Building  Objects  (MBOs)  and  Virtual  Modular 
Building  Objects  (VMBOs)  is  proving  highly  successful.  The  intent  is  to  reduce  both  cycle  time 
and  cost  by  increasing  the  reuse  of  existing  designs. 

Though  the  pilot  focus  changed  due  to  contractual  changes  to  the  MP-RTIP  program,  the 
initial  design  reuse  evaluation  is  an  indication  of  how  successful  Northrop  Grumman’ s  focus  on 
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design  reuse  has  become.  The  original  MP-RTIP  exciter  design  was  evaluated  just  after  the 
proposal  and  was  demonstrated  to  contain  45  percent  assembly  reuse  and  approximately  70 
percent  component  reuse  based  upon  the  Common  Radar  Modules  design  approach.. 

2.4.8  Software  Reuse 

A  searchable  software  reuse  library  was  developed  and  deployed.  The  library  is  expected 
to  continue  to  grow  and  evolve  using  Northrop  Grumman  funds. 

2.4.9  Design  for  Next  Generation 

Design  for  next  generation  has  been  demonstrated  successfully  as  a  part  of  the  original 
Modular  Building  Object  (MBO)  approach.  Design  engineering  was  reorganized  using  a 
functional  product  line  approach  in  both  engineering  and  manufacturing.  New  design  begins 
with  the  premise  that  the  design  will  be  reused  across  platforms  and  must  be  architected  in  a 
manner  to  enable  upgrades  and  maintenance.  This  is  documented  in  internal  procedures  and  is 
addressed  during  product  and  program  reviews. 

2.4.10  Preferred  Supplier  Meetings 

Commodity  Design  Teams  have  continued  to  hold  SPL  reviews  and  technology 
exchanges  with  preferred  suppliers.  Northrop  Grumman  provides  the  preferred  suppliers  with 
business  forecasts  and  anticipated  design  needs.  The  preferred  suppliers  provide  Northrop 
Grumman  with  standard  parts  lists  recommendations,  technology  forecasts  and  technical 
expertise. 

2.5  Recommendations  &  Design  Guidelines 

As  noted  in  the  previous  subparagraphs  of  this  section,  design  rules  and  design  guidelines 
were  created  and  design  reviews  implemented  to  track  compliance. 

2.6  Georgia  Tech  subcontract  -  physics  of  failure  models 

Over  the  last  decade,  IC  packaging  technologies  have  evolved  considerably  to 
accommodate  the  advances  made  in  the  semiconductor  industry.  Technological  advances  made 
in  the  semiconductor  industry  have  been  driven  primarily  by  the  commercial  market  needs. 
Meantime,  the  military  electronics  market  share  by  percentage  of  the  total  market  has  declined 
significantly,  and  is  estimated  to  be  less  than  1%  of  the  semiconductor  market  in  the  dollar  sale 
amount.  COTS  components  are  designed  and  developed  for  the  applications,  which  have  a 
different  set  of  product  life  requirements  (i.e.  long-term  reliability),  operating  and  storage 
environmental  requirements,  and  criticality  of  the  performance  reliability  than  the  military 
avionics  applications. 

The  challenges  associated  with  integrating  the  COTS  components  in  the  military  avionics 
applications  are  two-fold:  1)  the  long-term  reliability  when  subjected  to  the  harsh  environmental 
conditions  typical  of  the  military  avionics  applications,  and  2)  relatively  rapid  obsolescence  of 
the  COTS  components.  Some  of  the  COTS  components  are  robust  enough  so  that  they  can  be 
successfully  used  in  the  high  performance  military  avionics  applications.  Traditionally,  standard 
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accelerated  environmental  tests  are  performed  to  assess  and  ascertain  the  long-term  reliability  of 
the  COTS  components  to  utilize  them  in  the  military  applications.  Traditional  time  consuming 
test  and  qualification  approach  is  rendering  itself  inefficient  as  the  semiconductor  packaging 
technologies  are  advancing  rapidly  to  accommodate  the  fast  evolving  sub-micron  semiconductor 
technologies.  A  virtual  qualification  approach  based  on  physics  of  failure  needs  to  be  developed 
to  manage  integration  and  obsolescence  of  the  COTS  components  in  the  military  applications 

In  the  Commercial  Parts  Obsolescence  Management  (CPOM)  program,  physics  of  failure 
based-methodology  was  developed  to  enable  virtual  qualification  of  the  COTS  components  at 
the  zero  level  (i.e.  semiconductor  -  integrated  circuit  level)  and  at  the  second  level  of  packaging 
(i.e.  interconnect  level).  The  physics  of  failure  based  virtual  qualification  of  COTS  components 
methodology  was  accomplished  by  using  a  three  prong  approach,  which  consisted  of 
development  and  validation  of  electrical  models,  thermomechanical  models,  and  characterization 
of  the  commonly  used  electronic  packaging  materials  over  a  wide  range  of  environmental 
conditions.  The  electrical  models,  addressing  the  likely  failure  mechanisms  attributable  to  the 
failures  of  micron  level  and  sub-micron  level  semiconductor  packaging  were  compiled  in  the 
graphical  user  interface  (GUI)  format.  The  user-friendly  GUI  format  would  enable  a  larger 
engineering  community  in  the  industry  to  use  the  physics  of  failure  based-models  for  virtual 
qualification  of  the  COTS  components.  Likewise,  parametric  thermomechanical  models, 
addressing  the  failure  mechanisms  attributable  to  the  failures  at  the  second  level  (i.e.  an  IC 
package -to-PWB  interconnect  level)  packaging  were  compiled  in  the  GUI  format.  A  database  of 
material  properties  of  widely  used  electronic  packaging  materials  over  a  wide  range  of 
environmental  conditions  was  compiled  such  that  it  can  be  readily  integrated  into  the 
commercially  available  analysis  software  tools  such  as  ANSYS.  Additionally,  an  alternate 
accelerated  thermal  cycling  approach  based  on  the  physics  of  failure  was  proposed  to  optimize 
the  duration  of  the  accelerated  environmental  tests.  The  models  also  facilitate  evaluation  of 
reliability  enhancement  approaches  such  as  the  use  of  an  underfdl  at  the  second  level  packaging. 
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2.6.1  Electrical  Models 

A  wide  range  of  digital,  analog,  and  mixed-signal  integrated  circuit  (IC)  devices  (i.e. 
application  specific  IC-ASIC,  processors),  incorporating  sub-micron  level  semiconductor 
fabrication  technologies,  was  reviewed  to  determine  the  associated  failure  mechanisms  and 
modes.  Based  on  the  reviews,  electromigration  (EM,  hot  carrier  (HC),  and  electro  static 
discharge  (ESD)  were  identified  for  modeling.  GUI  based,  application  specific  IC  reliability 
assessment  tool  (ARET),  which  included  models  for  EM,  HC  and  ESD  was  developed  and 
validated. 


Figure  2-2  ASIC  Qualification  Methodology 

•  Developed  physics  of  failure  (POF)  based  reliability  models  for  mixed  signal  ASICs 
consisting  of  digital  as  well  as  analog  devices 

•  Developed  methodology  for  determining  the  failure  rates  and  expected  lifetimes  of 
commercially  available  mixed  signal  ASICs  based  on  POF 

•  Developed  POF  based  tool  which  includes  models  of  failure  mechanisms  most  likely 
to  cause  failures  when  the  COTS  devices  are  operated  in  the  typical  military 
application  environment 

o  ARET 

•  Uses  hierarchical  analysis  approach 

o  Behavioral  models  allow  propagation  of  electrical  stress  factors  at  the  circuit 
input  to  sub-modules  and  lower  level  building  blocks  -  top  down 
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o  Stress  factor  are  combined  with  the  physics  of  failure  models  to  assess  the 
performance  degradation  due  to  specific  failure  mechanism 

o  The  performance  degradation  is  propagated  bottom-up  through  hierarchy  to 
assess  the  performance  degradation  and  time  to  failure  at  the  higher-level  and 
the  system  level 

o  The  tool  accepts  net-lists  from  commercially  available  design  tools  such  as 
Cadence 


Figure  2-3  ARET  Output  for  Hot  Carrier  Failure  Mechanism 


Figure  2-4  ARET  Output  for  Electromigration  Failure  Mechanism 


•  Tool  Validation 

o  Various  test  structures  were  designed,  fabricated,  and  tested,  leveraging  a 
program  at  Boeing  for  the  model  validation 

•  AIM  C5N  fabrication  process 
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Figure  2-6  Test  Structures  for  Electromigration 


Figure  2-7  Test  Structures  for  Hot  Carrier 
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2.6.2  Thermomechanical  Models 

BGA  packages,  representative  of  the  ones  likely  to  be  used  in  the  military  avionics 
applications,  were  selected  after  thorough  literature  review,  inputs  from  engineering  at  Northrop 
Grumman  Corporation,  and  inputs  from  the  industry  work  shops  for  modeling  and  model 
validation.  A  test  vehicle  assembled  with  the  selected  BGA  packages  was  subjected  to 
accelerated  environmental  tests  to  validate  the  thermomechanical  models. 


Figure  2-8  Thermomechanical  Reliability  Models  and  Tools 

•  Developed  parametric  physics  of  failure  based  reliability  models  for  representative 
BGA  packages  and  assemblies 

o  The  models  utilize  the  time  and  temperature  dependent  viscoelastic  and 
viscoplastic  material  behavior 

o  The  models  allow  package  geometry  parameters  as  variables 
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•  Models  facilitate  assessment  of  solder  joint  reliability  enhancement  approaches  such  as 
use  of  underfill 

o  Developed  guidelines  for  selecting  underfill  material  properties  for  optimal 
reliability  of  underfill  IC  packages 

•  Developed  qualification  guidelines  for  IC  packages  used  in  military  field-used  thermal 
conditions 

o  An  alternative  accelerated  thermal  cycle  profile  representing  corresponding 
damage  that  occurs  in  the  actual  field  conditions  was  proposed  to  optimize  the 
cycle  time 

■  Industry  recommended  accelerated  thermal  cycle  may  be  too  long  and  may 
not  truly  represent  the  damage  caused  in  actual  field  life 

•  Developed  automated  post  processing  methodology  for  life  prediction  and  user  interface 
reliability  tools  for  multi-package  systems 

o  Parametric  GUI  based  tool  allows  a  user  to  build  models  through  graphical  user 
interface 

o  Tool  allows  a  user  to  vary  a  number  of  package  construction  parameters 

■  Package  I/O  pitch 

■  Number  of  I/Os 

■  Package  substrate  thickness  and  material 

■  Solder  sphere  size 

■  Package  side  solder  sphere  pad  size 

■  PWB  side  pad  size 

■  Solder  sphere  material 

•  Solder  sphere  may  be  lead- free 
o  Tool  allows  multi-package  system 

■  Solder  joint  reliability  of  a  number  of  packages  of  varying  constructions  types 
and  sizes  may  be  evaluated  in  one  simulation 

•  Model  validation 

o  Validated  the  solder  joint  reliability  assessment  from  models  with  the  test  data 

■  A  test  vehicle  designed,  fabricated,  and  assembled  at  Northrop  Grumman 
Corporation  was  tested  under  accelerated  thermal  cycle  tests  for  over  4000 
thermal  cycles 
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2.6.3  Characterization  of  Electronic  Materials 

In  electronic  packaging,  materials  varying  in  material  properties  and  dimensions  by 
multiple  orders  of  magnitude  form  interfaces.  It  is  critically  important  that  commonly  used 
materials  in  COTS  components  be  characterized  over  the  environmental  conditions  typical  of 
military  avionics  applications. 

•  Multiple  interfaces  were  characterized  over  a  wide  temperature  range  and  material 
properties  were  determined 

o  Metal-metal-metal  interface 

o  Organic  printed  wiring  board  (PWB)  material-metal  trace-solder  mask 
interface 

•  Multiple  interfaces  were  characterized  to  determine  the  effect  of  crack-tip  plastic 
deformation  on  the  interfacial  fracture  toughness  through  a  four  point  bending  test 

o  alumina-underfill  interface 

•  Underfill  material  characterization 

•  Fracture  toughness  evaluation  of  several  interfaces 
o  Underfill-polyimide 

o  Underfill-aluminum 
o  Underfill-FR4 

•  Effect  of  surface  roughness  on  interfacial 
o  Aluminum-FR4  epoxy 

•  Effect  of  moisture  on  interfacial  fracture  toughness 
o  Underfill  A  -  FR4  substrate 

o  Underfill  B  -  FR4  substrate 

•  Material  Properties  Database 

o  A  database  of  material  properties  of  commonly  used  electronic  packaging 
materials  was  developed  -  properties  were  either  determined  experimentally, 
or  collected  from  literature 

■  Semiconductor  materials 

•  Polycrystalline  Silicon 

•  Single  crystalline  Silicon 

•  Polycrystalline  Gallium  Arsenide 
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•  Eutectic  Solder  (63Sn/37Pb) 

•  Eutectic  Solder  (62Sn/36Pb/2Ag) 

•  Eutectic  Solder  (60Sn/40Pb) 

•  Eutectic  Solder  (70Sn/30Pb) 

•  High  Temp  Solder  (5Sn/95Pb) 

•  Lead-Free  Solder  (96.5Sn/3.5Pb) 

•  Underfill/Encapsulant 

o  HYSOL®  CNB832-20 
o  HYSOL®  FP4530 
o  Loctite®3564 
o  Loctite®3565 
o  Loctite®3568 
o  Alpha  EL18-LS-5091 10 

•  Organic  PWB  materials 

o  Nelco  N4000-2 
o  IPC  4102/21  (FR4) 
o  NEMA  FR-4  Glass  Epoxy 
o  Thermount® 


2.7  Pilot  Program 

In  1Q02,  Northrop  Grumman  embarked  upon  a  selection  process  to  find  the  program  that 
would  be  the  best  match  for  the  CPOM  Pilot.  The  selection  process  was  documented  and  the 
MP-RTIP  Exciter  design  was  selected  and  AFRL  concurrence  received.  The  pilot  also  included 
a  separate  RADSS  effort  on  APG68  Block  30/40,  a  MOCA  evaluation  and  a  DVTG  evaluation. 

The  MP-RTIP  SPO  concurred  and  the  pilot  was  started  in  2Q02. 

In  January  2003,  MP-RTIP  implemented  ECP  01,  changing  the  type  of  aircraft  using  the 
radar  and  almost  doubling  the  antenna  area.  Transmitter  power  was  increased  and  reflected 
noise  was  increased.  As  a  result,  a  lower  noise  Exciter  was  needed  and  the  in  process  design  had 
to  be  restarted.  In  addition,  the  MP-RTIP  schedule  slipped  to  the  point  that  it  no  longer  provided 
a  good  match  to  the  CPOM  period  of  performance. 

The  RADSS  effort  on  Block  30/40  was  not  affected  nor  were  the  MOCA  evaluation  or 
the  DVTG  evaluation. 
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With  the  situation  changed  so  dramatically,  a  new  approach  to  documenting  the  cost 
avoidance  resulting  from  the  implementation  of  obsolescence  management  tools,  reliability 
tools,  corporate  initiative  tools  and  lean  corporate  practices  was  proposed  and  accepted.  The 
CPOM  pilot  focused  on  collecting  tool  and  process  metrics  across  multiple  active  programs. 
Reports  were  formatted  to  annualize  cost  savings  by  tool  or  by  process.  In  many  ways  this 
retargeting  of  the  pilot  program  benefited  the  overall  program  as  there  was  a  higher  volume  of 
data  to  evaluate  and  program  specific  schedule  dependencies  were  eliminated. 

2.7.1  PEM  Issues,  Packaging  Issues,  Design  of  the  Environment 

The  objective  of  this  task  was  to  measure  the  cost  and  schedule  impact  of  PEM  issues, 
Packaging  issues  and  Design  of  the  Environment  activities  undertaken  in  the  ACME  portion  of 
the  program. 

PEM  selection  criteria,  Packaging,  handling  and  Design  of  the  Environment  data  has 
been  incorporated  into  Northrop  Grumman  Integrated  Enterprise  Process  work  instruction 
E241 10.  This  is  the  guiding  procedure  for  Commodity  Design  Team  day  to  day  activities. 

The  leading  edge  microcircuit  technologies  currently  available  as  Plastic  Encapsulated 
Microcircuits  represent  a  significant  increase  in  device  capability.  When  properly  selected,  they 
also  represent  a  significant  decrease  in  material  cost  when  compared  to  similar  hermetic  devices. 
For  high  pin  count  devices,  the  hermetic  package  itself  can  cost  around  $100. 

In  their  paper  “Plastic-Encapsulated  Microcircuit  Reliability  &  Cost  Effectiveness 
Study”,  D.  Emerson,  E.  Hakim,  and  A.  Govind  derived  a  cost  comparison  between  hermetic 
integrated  circuits  and  plastic  encapsulated  microcircuits  that  indicates  an  average  6-fold 
decrease  in  material  cost  when  commercial  devices  are  used.  This  is  much  more  apparent  with 
the  lower  pin  count  devices  as  the  package  dominates  the  material  cost.  As  pin  count  rises,  the 
integrated  circuit  chip  cost  dominants  and  the  cost  ratio  decreases;  however,  many  of  the  large 
scale  integrated  circuits  are  available  only  in  the  PEM  format.  Earlier  Northrop  Grumman 
analysis  indicated  that  a  four  to  one  ratio  is  more  appropriate  when  accounting  for  the  internal 
costs  associated  with  qualification  and  handling  of  COTs  and  represented  a  conservative 
estimate  of  the  cost  impact  of  PEM. 

Similar  analyses  for  industrial  versus  military  passive  devices  have  yielded  mixed  results. 
JPL’s  COTSCON  presentation  concluded  that  “commercial”  resistor  arrays  cost  them  more  than 
twice  as  much  to  process  than  their  equivalent  military  devices.  Given  those  results  and  the 
relatively  low  cost  of  the  discrete  passive  devices,  that  analysis  is  omitted. 

Analysis  indicates  that  disallowing  the  use  of  COTs  in  the  MRPTIP  exciter  design  would 
have  a  material  cost  of  approximately  $20K  per  system  based  on  the  difference  in  pricing 
between  hermetic  and  non  hermetic  parts. 

A  lead  time  comparison  for  parts  planned  for  the  MP-RTIP  program  was  also  performed. 
55%  of  the  PEM  planned  for  MP-RTIP  are  not  available  in  hermetic  packages  and  a  lead  time 
comparison  is  not  germane.  Disallowing  their  use  would  have  required  the  use  of  hermetic 
hybrid  modules.  For  the  parts  that  are  available  in  both  hermetic  and  plastic  packages,  the  lead 
time  for  PEM  averaged  4.1  weeks  and  the  lead  time  for  the  hermetic  packaged  microcircuits 
averaged  8.3  weeks.  The  additional  delivery  time  for  the  hermetic  devices  is  not  significant 
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in  terms  of  system  hardware  delivery  and  is  easily  addressed  through  material 
requirements  planning. 

A  much  larger  cost  impact  can  be  derived  by  looking  at  the  parts  in  the  design  for  which 
hermetic  equivalents  don’t  exist.  It’s  logical  to  conclude  that  disallowing  the  unsealed 
microcircuits  would  lead  to  the  use  of  hermetic  multichip  modules.  A  ballpark  estimate  on 
design  cost  for  a  complex  hermetic  module  is  that  it  takes  seven  months  to  do  the  first  design 
turn  and  five  months  to  do  the  second  design  turn.  It’s  rare  to  stabilize  a  final  design  without  at 
least  two  design  turns.  Total  cost  for  such  a  module  would  be  in  the  range  of  $1.5M  as 
compared  with  the  industry  model  of  $700K  for  a  board.  If  we  assume  that  the  4.5  reused  boards 
in  the  original  proposal  couldn’t  be  used  the  redesign  cost  would  be  approximately  $3.1M.  If  we 
assume  that  PEM  aren’t  allowed  and  we  have  to  design  hermetic  hybrids,  we  most  likely  create 
additional  integration  and  replace  4.5  boards  with  3  hybrid  modules  at  a  cost  of  approximately 
$4.5M.  The  time  required  for  the  hybrid  redesign  is  approximately  1  year  assuming  that  all 
modules  proceed  in  parallel.  This  design  cycle  time  isn’t  significantly  different  from  the  board 
design  time. 

In  some  cases  environmental  requirements  preclude  the  use  of  PEM  without  additional 
design  of  the  environment.  The  most  typical  of  these  is  a  thermal  environment  such  that  junction 
temperatures  cannot  be  adequately  controlled.  The  work  done  on  CPOM  helped  to  define 
improved  thermal  management  practices  and  to  increase  awareness  of  the  issue.  Design  rules 
have  been  enhanced  and  incorporated  into  design  review  practices.  A  thermal  analysis  is 
performed  for  each  design  and  the  results  are  reviewed  by  reliability  engineering.  Problem  areas 
are  identified.  The  solution  may  be  as  simple  as  a  design  iteration  changing  part  selection. 
Approximately  50%  of  PEM  applications  in  harsh  environment  require  extraordinary  design  of 
environment  approaches. 

2.7.2  Tool  Development  and  Evaluation 

2.7.2.1  RADSS  -  APG-68  Block  30  Evaluation 

Northrop  Grumman  selected  the  F-16  AN/APG-68  Radar  Programmable  Signal 
Processor  (PSP)  as  the  pilot  program  to  execute  and  evaluate  RADSS.  The  PSP  contains  27 
unique  Circuit  Card  Assemblies  (CCAs)  with  upwards  of  100  obsolete  microcircuit  part  types. 
Because  of  the  extreme  degree  of  AN/APG-68  PSP  obsolescence  and  the  high  cost  to  resolve, 
NGC  has  been  no-bidding  RFQs  for  spare  hardware.  Northrop  Grumman  installed  the  RADSS 
tool,  defined  and  developed  a  DMS  decision  model,  collected  data  and  ran  analysis  of  tool 
capabilities  and  weaknesses. 

Seven  separate  analysis  scenarios  were  used  to  exercise  the  RADSS  tool.  The  following 
are  examples  of  those  analyses. 

Scenario  #1 

•  RADSS  choose  the  alternatives  with  the  highest  benefit/cost  ratio. 

•  No  constraints 

•  Solution  set  contained  an  alternative  for  all  27  CCA  styles  regardless  of  profitability 
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Scenario  #2 

•  Solution  set  to  only  contain  profitable  alternatives  from  scenario  #1 

•  Constrained  on  profit.  Excluded  all  alternatives  with  a  NPV  <  or  =  0 

•  Solution  contained  an  alternative  for  14  of  the  27  CCA  styles. 

Scenario  #3 

•  Solution  set  to  contain  only  profitable  alternatives 

•  Constrained  on  fiscal  DMS  budget 

•  RADSS  dropped  CCA  9  and  19  from  the  solution  set  to  climb  under  the  fiscal  DMS 
budget  constraint 

Scenario  #4 

•  Solution  set  to  only  contain  profitable  alternatives 

•  Constrained  on  resources.  Maximum  of  4200  engineering  hours 

•  RADSS  excluded  CCA  19  from  the  solution  set  to  meet  the  manpower  constraint 

•  Keep  recently  hired  Engineers  busy 

•  Constrained  on  4200  -  9000  engineering  hours 

•  Constrained  on  fiscal  DMS  budget 

•  Solution  contained  8  CCAs 

The  pilot  demonstrated  multiple  business  cases  for  profitability  on  otherwise 
obsolete  designs,  producing  a  win  win  situation  for  NGC  and  the  Air  Force.  The  analyses 
showed  that  RADSS  is  a  powerful  decision  support  tool  that  is  best  used  on  complex 
obsolescence  sets.  It  allows  the  user  to  make  decisions  based  on  decision  criteria  established  by 
the  organization  and  to  make  “what  if’  adjustments  to  the  data  to  optimize  the  ultimate  decision. 
Due  to  the  complicated  setup  and  model  definition,  the  tool  is  not  well  suited  for  small  scale 
obsolescence  decision  modeling. 

2.1.22  \2  LCM  -  Task:  Evaluate  ASPECT/i2  LCM  Tool 

LCM  (version  5.0)  AFRL  BAA  97-1 1-MLKT  is  currently  available  on  the  commercial 
market  as  a  production  product;  however,  it  requires  a  significant  software  system  upgrade 
(SRM)  in  order  to  implement.  Northrop  Grumman  has  negotiated  a  license  agreement  with  i2 
that  allows  the  use  of  the  i2  products  across  all  sector  sites. 

Northrop  Grumman  Electronic  Systems  migrated  additional  data  from  the  Rolling 
Meadows  and  Woodland  Hills  sites,  installed  the  software  and  trained  users.  This  is  a  first  time 
installation  of  the  tool  at  Woodland  Hills  and  an  upgrade  of  the  capabilities  at  Rolling  Meadows. 
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Utilizing  the  eDesign  version  of  i2’s  software,  the  life  cycle  management  data  has  been 
fully  integrated  into  day  to  day  component  and  bill  of  material  analysis  activities  across  the 
sector.  Due  to  some  issues  with  the  i2  software,  direct  oracle  joins  to  the  LCM  data  were  used  as 
a  means  of  accelerating  the  analyses  and  automating  portions  of  the  process.  In  addition,  i2’s 
LCM  data  and  critical  parameters  from  their  commercial  catalog  data  were  integrated  into  the 
Bill  of  Material  analysis  activity  utilizing  Titan’s  Poet  software. 

Northrop  Grumman  and  Lockheed  Martin  and  BAE  continue  to  engage  in  i2  data 
discussions  as  active  members  of  the  i2  users  group.  This  intercorporate  teaming  drove  i2  to 
correct  data  discrepancies  and  algorithm  differences  between  LCM  and  the  original 
Tactrac  database.  i2  announced  completion  of  this  database  synchronization  activity  at  the 
User  Group  meeting  in  October  03.  This  removes  a  major  obstacle  allowing  OEMs  to 
implement  the  SRM  tool  as  it  was  originally  intended. 

Northrop  Grumman  utilizes  i2  LCM  in  all  stages  of  emerging  designs:  Proposal,  Design, 
Manufacturing,  Sustainment,  Redesign,  MOCA  evaluation,  RADSS  evaluation,  day  to  day  parts 
selection  and  Common  Product  Team  design  activities.  It  is  integral  to  the  business  and  has  very 
wide  exposure.  Common  new  part  selection  and  qualification  procedures  were  agreed  upon  and 
workflows  updated  to  include  former  Litton  sites. 
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Listing  of  programs  impacted  in  2003  based  on  data  exports  from  the  i2  tool. 


Advanced  Technology  Transmitter 

Agile  Beam  Radar 

ALQ-131 

APG-68  V(X)M 

ASDS 

ASPJ 

AWACS 

Block  V 

BIB 

BAT 

Comanche 

DUS&T 

ELMO 

F15K 

F-22  Falcon  Edge 

Flat  Bed  Sorter 

ICAP 

IFTS 

Joint  Strike  Fighter 

Longbow  Missile 

Longbow  Fire  Control 

LR100 

Manifest 

Modar 

MSTRS 

PDF  Lot  2 

Pod  Radar 

RTIP 

SBIRS 

SBR 

SPQ9B 

TPS-70  Solid-State 

2.7. 2.3  DVTG/VectorGen  Evaluation 

Developed  a  Rosetta  model  for  a  Northrop  Grumman  frequency  synthesizer  module  and 
used  the  model  as  an  input  to  the  DVTG/VectorGen  tool,  successfully  generating  accurate  test 
vectors. 

First,  a  few  definitions. 

Rosetta  is  an  emerging  high-level  description  language  for  specifying  systems  that  explicitly 
provides: 

•  Support  for  domain  specific  modeling 

•  Support  for  modeling  cross  domain  interactions 

•  Support  for  defining  and  combining  models  of  systems  and  system  components 

•  Support  for  modeling  and  analysis  at  high  levels  of  abstraction 

•  Support  for  specifying  constraints  and  performance  requirements  for  the  system  and  system 
components  in  a  top-down  manner 

•  Support  for  requirement  verification  as 

•  A  facet  is  a  model  representing  one  perspective  of  a  system 

-  One  particular  point  of  view 
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-  One  particular  abstraction  level 

•  A  domain  is  a  semantic  system  for  defining  facets 

•  A  system  is  described  by: 

-  Defining  and  composing  its  various  facets 

-  Defining  and  composing  its  various  sub  the  system  description  evolves 

The  approach  was  to  describe  the  frequency  synthesizer  using  multiple,  interacting 
models  and  then  combine  the  models  into  a  single  functional  description  from  which  test  vectors 
could  be  electronically  generated.  This  simplified  the  development  of  the  Rosetta  code  and 
made  individual  models  easier  to  understand  and  to  test. 


Digital 


Figure  2-9  Synthesizer  Parent  Program 


Conclusions 

Rosetta 

Rosetta  succinctly  captured  the  functional  perspectives  of  the  FPGA,  RF  and  Power 
models  of  the  Synthesizer  and  language  similarities  made  Rosetta  easy  to  grasp.  Rosetta  is  in 
need  of  an  Operational  Subset.  In  its  present  form,  it  is  quite  adequate  for  design  capture; 
however,  it  doesn’t  permit  execution  of  code  that  might  be  needed  for  debugging  or  other 
purposes.  An  operational  subset  would  permit  execution  by  allowing  Rosetta  specification 
language  to  be  interpreted  as  a  set  of  instructions  that  can  be  interpreted  as  a  program  the  same 
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way  as  C  or  Haskell.  The  Interpreter  would  also  have  to  be  developed  to  evaluate  expressions 
and  produce  values  for  input  parameters 

Vector  Gen  Tool 

VectorGen  was  able  to  verify  models  with  correct  output  data.  The  tool  was  straight 
forward  and  easy  to  navigate.  In  its  present  iteration,  VectorGen  doesn’t  handle  nested  models. 


2.1. 2 A  MOCA  Evaluation  PilotLoaded  program  data  from  a  sustainment  program  into  MOCA 
to  effectively  turn  back  the  clock  and  compare  the  MOCA  predictions  to  known  occurrences. 


-  5  years  of  Obsolescence  Management  Data 

-  Tactech  predictive  data 

-  Historical  DMS  notices 

-  Obsolescence  mitigation 
actions 


Plan 


Actual 


MOCA 

output 
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The  pilot  program  ran  MOCA  as  if  it  were  1998 
-  Compared  results  to  observed  actuals 


Northrop  Planned  Redesign  Activity 

•  Planned  program  life  was  assumed  to  be  20  years 

•  Quarterly  DMS  analyses  were  performed 

•  DMS  issues  were  addressed  to  reach  the  next  redesign 

•  A  three  year  redesign  schedule  was  chosen 
Actual  Redesign  HistoryMajor  redesign  was  later  than  planned 

-  Production  was  low  and  funding  was  limited 

•  Parts  buys  or  alternate  parts  were  desired  to  avoid  redesigns 

-  2000:  1  alternate  part 

-  2001 :  7  parts  buys  and  1  alternate  part 

-  2002:  9  parts  buys 

•  Parts  buys  were  only  allowed  out  to  Lot  3 

•  Qualification  activities  also  negated  unnecessary  redesigns 

•  Contract  essentially  forces  1  year  look  ahead 

•  MOCA  indicates  a  3.5  year  look  ahead  is  optimal 

Obsolescence  Comparisons.  Having  reviewed  the  data  from  the  sustainment  program,  we  were 
able  to  provide  an  assessment  of  the  DMS  predictive  data  tools  as  well. 

•  TacTech  and  CALCE  are  pessimistic 

•  The  dataset  consisted  of  95  parts 

•  Correlation  is  poor 

-  CALCE  is  driven  by  technology 

•  MOCA  needs  accurate  data 
MOCA  Assessment 

•  Integration  to  POET  and  PRICE  works  well 

•  MOCA  output  looked  reasonable  despite  data  limitations 

-  However 

-  Contract  funding  drives  actuals 

-  To  maximize  benefit  of  MOCA,  contracts  should  better  recognize  and  fund 
obsolescence  mitigation 

-  Data  sources  are  too  pessimistic 

•  MOCA  display  is  clear 
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-  Detail  reporting  could  be  improved 

•  Organizes  a  refresh  plan  in  a  manner  consistent  with  life  cycle  cost 

•  UMD  has  developed  a  unique  tool  with  significant  potential  to  impact  life  cycle  costs 

2.7.2.5  Poet/Workflow  Demonstration 

This  was  a  multi  part  task  that  began  with  the  evaluation  of  the  Rosetta  language, 
continued  through  the  development  of  the  Poet  software  suite  and  finished  with  a  live 
demonstration  of  a  Poet  interactive  workflow  interfacing  with  multiple  design  tools  and  data 
sources. 

Rosetta 

Rosetta  models  were  created  to  represent  the  mathematical  results  of  the  investigations 
performed  for  Physics  of  Failure  modeling.  These  models  are  also  appropriate  for  use  in  writing 
and  verifying  systems  specifications  in  the  areas  covered  by  the  models.  Titan  Systems 
Corporation  personnel  coordinated  with  the  Northrop  Grumman  investigation  team  conducting 
the  analyses  to  be  modeled  in  SLDL.  The  SLDL  models  present  the  results  produced  by  the 
investigations  in  a  manner  that  will  enable  systems  designers  to  specify  their  designs  with  strict 
constraints  on  the  factors  controlling  the  success  of  including  commercial  dies,  designs,  and 
parts  on  military  substrates.  SLDL  models  completed  as  a  part  of  this  effort  included:  Corrosion 
model  based  on  the  Peck  model,  Stress  failure  model  based  on  the  Coffin-Manson  model  for 
predicting  end  of  life  due  to  thermal  stresses,  Mean  Time  Between  Failure  Model  (MTBF)  using 
the  RAC  model  augmented  with  physics  of  failure  techniques  and  field  reliability  data  to 
electronically  track  changes  that  might  impact  reliability,  Cooling  Schema  Model  permitting  the 
specification  of  levels  of  convection  and  conduction  cooling,  both  for  "cavity  up"  and  "cavity 
down"  BGA  packages,  Dendritic  Growth  model  to  capture  the  mathematical  representation  of 
the  dendritic  growth  between  fine  pitch  mounting  pads  for  printed  wiring  boards,  Thermal 
Expansion  Model  for  design  rule  checking  of  electronic  Bills  of  Materials  for  potential  thermal 
mismatch  and  an  Intellectual  Property  Model  to  demonstrate  the  capture  of  module  level  design 
and  test  vector  generation  data  utilizing  the  Rosetta  language. 

Proved  out  Rosetta  in  the  context  of  both  design  and  requirements.  Proved  Rosetta  was 
easy  to  learn  and  use  by  new  designers  not  familiar  with  system  level  languages.  Demonstrated 
the  feasibility  of  using  Rosetta  models  in  combination  with  Poet  for  the  automation  of  design 
rules  checking 

All  of  the  above  benefits  served  to  provide  constructive  feedback  to  the  Rosetta 
development  community  in  order  that  future  language  and  tool  development  would  be  sure  to 
incorporate  lessons  learned  from  CPOM  experience,  thus  making  the  language  and  tools  more 
usable  and,  in  turn,  more  commercially  viable. 

Poet  Development 

The  primary  focus  of  Titan  during  the  CPOM  program  was  the  development  of  Poet.  Poet 
is  a  software  tool  that  enables  designers  to  efficiently  manage  data,  tools,  design  rules  checks 
(DRCs),  and  processes  in  a  centralized  and  coherent  manner.  With  regard  to  the  objectives  of 
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CPOM,  Poet  provides  a  commercially  viable  solution  to  facilitate  the  redesign  of  obsolete 
assemblies  by: 

•  Enabling  process  management  via  workflow 

•  Providing  information  management 

•  Providing  product  management  (design  data  and  design  validation  and  verification) 

•  Enabling  tool  invocation 

•  Providing  an  intuitive,  web-enabled  interface 

Titan  was  able  to  draft  initial  requirements,  design,  develop  and  test  Poet  during  the 
CPOM  program.  The  testing  was  completed  by  installing  Poet  at  a  Northrop  Grumman  facility 
and  by  processing  design  data  from  a  real  Northrop  Grumman  assembly.  Afterward,  a  series  of 
demonstrations  was  presented  to  both  Northrop  Grumman  and  the  Air  Force  to  verify  that  Poet 
was  indeed  working  as  described.  Some  of  the  features  presented  at  the  demonstrations  includes 


Figure  2-10  A  Summary  View  of  Multiple  Projects 
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Figure  2-11  An  Overview  for  a  Single  Project 
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Figure  2-12  Data  Files  That  Constitute  the  Project 
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Figure  2-13  Design  Rule  Checks  to  be  Run  Against  Project  Data 
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Figure  2-14  Tools  to  be  Used  to  Accomplish  Project  Tasks 
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Benefits  Realized 


Many  benefits  have  been  realized  through  the  development  of  Poet.  Some  of  the 
highlights  include 

•  Integrating  heterogeneous  design  information  and  tools 

•  Managing  the  information  flow  through  a  design  process 

•  Promoting  collaboration  and  insight  throughout  design  process 

•  Providing  the  ability  to  validate  and  verify  new  design  form,  fit,  and  function  against 
legacy  design 

•  Facilitating  repeatability  in  design  process 

•  Providing  auditable  process  conformance  (CMMI) 

•  Providing,  through  use  of  Rosetta,  consistent  method  for  design  capture 

Driving  the  Design  Process  Using  Poet 

To  assess  the  commercial  viability  of  Poet,  Titan  undertook  a  follow-on  task  to 
demonstrate  how  Poet,  in  conjunction  with  a  commercially  available  workflow  tool  (Mentor’s 
WorkXpert),  could  be  applied  to  drive  a  real  world  design  process  at  Northrop  Grumman.  The 
project  was  defined  to  address  four  key  elements  in  driving  a  process: 

1.  Define  and  implement  a  design  process  using  the  workflow  tool.  For  this  task,  an 
established  Northrop  Grumman  design  process  would  be  captured  using  the  workflow 
tool.  One  of  the  steps  of  the  process,  BOM  Analysis,  would  serve  as  a  test  case  for 
automating  a  process  step.  A  bi-directional  interface  between  the  workflow  tool  and 
Poet  would  need  to  be  established  to  facilitate  the  brokering  of  information  between 
the  two. 

2.  Automate  BOM  Comparison  and  Analysis.  Poet  would  be  used  to  resolve  the 
differences  among  multiple  Bill  Of  Material  sources  that  include  CAD  design  data, 
components  databases,  and  web-enabled  interfaces  to  databases.  The  analysis  would 
consist  of  twenty-five  tests  that  ranged  from  determining  critical  lead  times  to 
identifying  components  with  Diminishing  Manufacturing  Source  (DMS)  issues. 

3.  Collection  of  Cross  Project  Metrics.  Poet  would  be  used  to  collect  predefined 
metrics  from  the  workflow  tool.  The  workflow  tool  itself  is  capable  of  only  reporting 
on  a  single  design  project  at  a  time.  Poet  was  to  provide  the  added  power  of  being 
able  to  pull  data  from  multiple  projects  into  a  single  reporting  mechanism,  thus 
enabling  a  designer  or  manager  the  ability  to  view  a  snapshot  of  the  status  of  all 
projects  in  a  single  report. 

4.  Centralize  project  tracking.  A  web-based  producibility  checklist  was  used.  The 
purpose  of  this  task  was  to  take  an  Excel  spreadsheet  that  was  used  for  project 
tracking  purposes  and  to  web-enable  it  in  a  manner  that  allowed  users  access  to,  and 
modify  as  necessary,  its  information  without  creating  a  need  to  pass  around  multiple 
copies  of  the  spreadsheet. 
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Accomplishments 

The  Northrop  Grumman  and  Titan  team  was  able  to  meet  the  goals  of  demonstrating 
capabilities  to  drive  processes  by  Defining  and  implementing  a  design  process  workflow  using 
the  Mentor  WorkXpert  tool.  Automating  a  step  of  the  workflow,  BOM  Comparison  and 
Analysis,  using  a  standard  test  interface  that  could  be  employed  in  the  development  of  additional 
automated  analyses.  Twenty- five  different  BOM  Analysis  tests  using  four  different  BOM  data 
sources  were  implemented.  Gathering  and  reporting  of  eight  key  project  metrics  from  the 
WorkXpert  workflow.  Developing  and  implementing  a  web-based  tool  for  project  tracking  that 
assumes  the  role  previously  held  by  an  Excel  spreadsheet.  These  four  tasks  (workflow 
development,  BOM  analysis,  cross-program  metrics  reporting,  and  producibility  checklist) 
provide  key  elements  necessary  to  drive  processes  in  a  repeatable  manner. 
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Figure  2-15  Producibility  Checklist  Main  Page 
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Figure  2-16  Producibility  Checklist  Automated  Action  Items 
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Figure  2-17  Admin  Page  Project  Control 


Figure  2-18  Admin  Page  Users  Control 

The  Producibility  Checklist  provides  a  centralized  mechanism  for  tracking  project  status 
by  providing  features  such  as  (from  upper  left)  facilitating  signoffs  across  all  phases  of  the 
design  cycle,  capturing  and  tracking  action  items,  managing  user  access  and  capabilities  for 
individual  users,  and  managing  the  creation  and  archiving  of  individual  projects. 
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Benefits  Realized 

The  key  benefit  achieved  by  the  combination  of  the  four  task  areas  is  a  capability  to  drive 
processes  in  a  consistent  and  repeatable  manner.  This  benefit  is  achieved  as  a  result  of  the 
combination  of  benefits  achieved  for  each  of  the  individual  tasks.  These  benefits  include: 

Workflow  Implementation 

•  Documentation  of  process 

•  Repeatability  of  process 

•  Automation  of  process  steps  where  appropriate 

•  Ability  to  track  project  based  on  tangible  metrics 

BOM  Analysis 

•  BOM  analysis  is  automated 

•  BOM  analysis  is  repeatable 

•  BOM  discrepancies  caught  earlier  in  the  process,  thus  avoiding  costly  rework 

•  Recurring  engineering  functions  have  been  reduced  to  a  single  NRE  cost 

•  Test  data  preserved  and  maintained 

•  Tests  are  developed  through  a  common  interface  that  speeds  development  of  new 
analyses  tests 

Metrics  Capture 

•  Centralized  metrics  report 

•  Metrics  based  on  measurable  state  of  workflow 

•  Metrics  capture  across  programs  and  not  just  for  a  single  oneReduces  configuration 
challenges  of  maintaining  a  single  spreadsheet  amongst  the  entire  design  team 

•  Provides  access  to  producibility  checklist  to  entire  design  team,  as  well  as  managers 
and  others  with  vested  interest  in  project  status 

•  Extensible  to  include  additional  features 

Conclusion 

Titan  has  been  able  to  develop  a  commercially  viable  services  offering  based  on  the 
development  of  Poet  and  its  capabilities  for  driving  processes  as  previously  described.  This 
commercial  viability  is  demonstrated  in  the  ongoing  discussions  between  Northrop  Grumman 
and  Titan  to  further  institutionalize  the  results  of  the  CPOM  program  at  Northrop  Grumman.  As 
the  development  of  a  commercially  viable  offering  was  a  key  objective  for  Poet  in  the  CPOM 
program,  the  program  itself  can  be  considered  a  success  in  achieving  this  objective. 
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Figure  2-19  Bill  of  Material  Analysis  Automated  using  Poet  and  WorkXpert 
2.7  Improved  Policies,  Practices,  and  Procedures 

Common  Product  Teams  established  design  rules,  value  stream  mapped  design  processes  and 
established  a  detailed  design  review  process  resulting  in  ECA  cycle  time  quotes  down  23.5%  ( 
~175  designs),  MCM  cycle  time  quotes  down  12  %  (~  15  designs)  with  Structural  system 
design  cycle  time  quotes  steady.  With  some  simplifying  assumptions  this  translated  into 
~$1 1  M/year  CPT  design  cycle  time  reductions 

Commodity  Design  Teams.  Commodity  Design  Teams  processed  6855  New  Part  Requests  in 
2003  using  the  i2  workflow  and  supporting  data.  Of  those,  6027  were  evaluated  and  approved  as 
submitted,  722  were  rejected  and  replaced  with  existing  or  standard  component  selections  (722  x 
$9400  =  $6.8M),  4624  part  numbers  revised  or  corrected.  ($1.86M/yr  in  revision  notice 
avoidance)  totaling  ~  $8.7M/year  across  23  major  program  developments. 

Enhanced  reliability  modeling.  Reliability  prediction  procedures  updated  to  reflect  tool  and 
field  data  evaluations. 
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Documented  >$22M  annual  cost  reduction  from  processes  and  tools 

Improved  thermal  design  rules  implemented  with  a  few  modeled  and  electronically  automated 
using  the  Poet  software. 

Improved  material  handling  through  electronic  automation  of  data  flow  between  the  PDM 
system  and  the  manufacturing  floor.  Material  handling  codes  developed  and  implemented  to 
display  shelf  life,  moisture  sensitivity,  packaging  for  manufacturability.  Data  is  input  through 
the  New  Part  Review  process,  evaluated  through  the  design  review  process  and  displayed  on  the 
manufacturing  Bill  of  Material. 

Improved  manufacturability  through  incorporation  of  manufacturing  rules,  lessons  learned  and 
packaging  requirements  in  the  New  Part  Review  and  Design  Review  processes. 

Enabled  COTS  technologies  by  establishing  qualification  thresholds  and  thermal  management 
processes. 

Ledger  stocking  program.  Established  inventory  stocking  procedure  that  links  Standard  Parts 
Lists  and  high  use  components  to  ledger  stores. 

Supplier  Managed  Inventory  Program.  Commodity  Design  Teams,  procurement  data  and  i2 
data  used  to  establish  a  supplier  managed  floor  stocking  program  for  high  use,  low  cost 
components  thereby  reducing  procurement  costs  and  manufacturing  cycle  time. 

ECAD  library  process  improvements.  Electronically  integrated  ECAD  model  creation 
process  with  the  New  Part  Review  process  decreasing  the  number  of  models  created  for  parts  not 
suitable  for  production.  This  had  the  added  effect  of  getting  the  new  part  reviews  to  occur  much 
earlier  in  the  design  process  and  enabling  electronic  data  transfer  from  ECAD  tools  to  the 
Product  Data  Management  system. 

2.8  Business  Practices  Roadmap 

•  Implement  a  control  process  for  the  review  of  components  and  assemblies  intended  for 
use  in  new  designs.  From  a  Northrop  Grumman  standpoint,  this  was  a  primary  goal 
for  the  Policies,  Procedures  and  Practices  task  on  the  CPOM  program.  Many  new 
contracts  now  require  OEMs  to  propose  internal  parts  selection  and  standardization 
processes  or  be  subjected  to  externally  imposed  controls.  Industry  attempts  to 
document  such  procedures  have  resulted  in  such  documents  as  EIA-4899,  Standard  for 
Preparing  an  Electronic  Component  Management  Plan,  EIA-933,  Standard  for 
Preparing  a  COTS  Assembly  Management  Plan,  FAA  COTS  Risk  Management 
Guide,  EIA  SSB-1,  Guidelines  for  Using  Plastic  Encapsulated  Microcircuits  and 
Semiconductors  in  Military,  Aerospace  and  Other  Rugged  Applications 

•  Establish  a  formal  New  Part  Request  (NPR)  routing  and  approval  process.  The  i2 

Component  and  Supplier  Management  tool  works  well  as  an  automation 
methodology. 

•  Link  to  CAD  tools.  Any  Component  or  COTS  management  program  must  be  linked 

to  the  design  tools  if  it  is  to  be  effective.  Without  such  a  link,  controls  occur  too  late 
in  the  design,  adding  cost  and  cycle  time  to  the  program  development 
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•  Establish  component  test  methodologies  and  qualification  thresholds  utilizing  industry 

models  and  physics  of  failure  analysis. 

•  Utilize  cross  functional  reviews  to  assure  that  design,  quality,  reliability, 

manufacturing  and  business  needs  are  met.  Commodity  Design  Teams  were  the 
implementation  methodology  at  Northrop  . 

•  Establish  and  publish  design  rules  and  standard  design  processes.  Common  Product 

Teams  were  established  to  address  core  design  processes,  design  rules  and  design 
reuse. 

•  Design  Reviews.  Detailed  technical  design  reviews  and  program  reviews  are  critical 

to  program  success.  Documentation  of  these  reviews  and  associated  action  items 
provide  the  necessary  discipline  for  successful  execution. 

•  Update  reliability  prediction  methodologies  and  selection  rules  to  accurately  address 

COTs  technology.  Northrop  Grumman  reliability  evaluated  existing  models  and  tools 
and  compared  the  results  to  field  data  to  determine  the  most  accurate  prediction 
approach  and  to  established  updated  procedures. 

•  Make  material  costs  and  manufacturing  costs  visible  to  the  designer.  Many  companies 

charge  designers  with  Design  to  Cost  goals.  Making  basic  material  and 
manufacturing  costs  available  is  a  must  if  cost  is  a  requirement. 

•  Establish  a  referee.  NPR  elevation  procedures  must  be  in  place  to  settle  disputes 

between  the  review  teams  and  the  program  design  teams.  In  Northrop  Grumman’ s 
implementation,  an  override  request  for  a  rejected  NPR  must  go  to  the  Vice  President 
of  Engineering  and  Manufacturing  for  mediation. 

•  Make  the  preferred  parts  easy  to  find.  Standard  parts  lists  must  be  made 

parametrically  searchable  and  maintained  in  an  electronically  accessible,  controlled 
format.  The  i2  Component  and  Supplier  Management  tool  is  a  good  means  of  doing 
that  and  offers  a  great  deal  of  supporting  data  like  detailed  component  parameters, 
data  sheets  and  DMS  data. 

•  Focus  business  influence.  Where  design  complexity  does  not  support  the  use  of 

standard  components,  design  and  procurement  activity  should  be  focused  on  a 
relatively  few  preferred  suppliers. 

•  Measure  your  progress  carefully.  The  old  adage  that  you  get  what  you  measure 

applies  when  developing  metrics.  Some  do’s  and  don’ts  with  respect  to  metrics. 

•  Establish  component  test  methodologies  and  qualification  thresholds  were  established 
utilizing  industry  models  and  physics  of  failure  analysis. 

•  Percentage  of  Standard  Part  List  (SPL)  parts  on  new  parts  listings.  This  is  an 
indication  of  both  the  robustness  of  the  SPL  and  the  degree  to  which  designers  are 
looking  at  it.  If  improperly  interpreted,  tracking  this  metric  can  lead  to  a  large,  out  of 
control  list  of  standards. 
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•  Percentage  of  SPL  parts  on  maturing  programs.  This  is  an  indication  of  the  growth  of 
the  SPL  towards  the  design  direction.  Program  usage  is  clearly  a  criterion  for  SPL 
addition. 

•  Number  of  times  the  SPL  is  accessed.  Measures  the  overall  program  success.  Low 
access  rates  can  indicate  either  a  lack  or  training  or  an  inadequate  SPL. 

•  Percentage  of  SPL  parts  on  Corporate  Contract.  Prenegotiation  of  purchase  orders 
for  standard  parts  reduces  procurement  costs,  reduces  procurement  cycle  time  by 
enabling  automation  of  purchase  orders  and  encourages  designers  to  use  the 
standards. 

•  Monitor  SPL  activity:  Number  of  changes,  SPL  reviews  (DMS,  Cost. . .)  In  order  to 
be  viable,  an  SPL  must  be  monitored  and  kept  up  to  date.  The  SPL  itself  should  be 
handled  like  a  program  bill  of  materials  and  design  reviews  performed  routinely  with 
both  designers  and  suppliers. 

•  NPR  cycle  time:  New  Part  Request  review  cycle  time  must  be  held  to  a  level  that  can 
be  tolerated  by  design  and  production 

•  Number  of  NPRs.  Simply  counting  new  part  requests  and  holding  designers  to  a 
specific  limit  could  result  in  designs  using  older  technologies 

•  Number  of  new  parts  introduced.  This  is  one  of  the  easier  metrics  for  assessing  cost 
impact.  Northrop  Grumman  uses  the  Coopers  and  Lybrand  data  for  cost  modeling 
purposes 

•  Standard  Part  Usage  must  be  enabled  by  getting  SPL  parts  into  Engineering 
Storerooms,  onto  corporate  contracts  and  into  ledger  or  supplier  managed  inventory 
programs 

•  Number  of  design  revisions.  This  is  another  common  metric  that  can  backfire  if  not 
used  carefully.  Grading  engineers  on  the  number  of  revision  notices  to  a  Bill  of 
Materials  can  result  in  revision  notices  being  held  for  batch  processing.  If  at  a 
production  stage,  this  can  result  in  hardware  or  materials  that  don’t  meet  the 
application  requirements.  Reasons  for  revision  notices  must  be  taken  into  account. 

2.9  Hurdles  to  Design  Process  Optimization 

The  largest  impediment  for  programs  moving  from  legacy  military  design  practices  to 
optimized  design  practices  is  in  recognizing  the  need  for  process  and  tool  enhancements. 
Performance  contracts,  the  use  of  COTS  in  harsh  environments,  accelerated  obsolescence  of 
technologies  and  lean  business  objectives  demand  critical  changes  in  design  practices.  As  is  said 
about  many  illnesses,  admitting  the  problem  is  half  the  battle.  Once  a  program/corporation 
acknowledges  the  issues,  executive  level  management  support  is  required  to  drive  the 
documentation  of  repeatable  processes,  the  automation  of  those  processes  and  the 
implementation  of  a  metrics  monitoring  program.  Lacking  top  level  support,  internal  politics 
and  inadequate  communications  will  severely  inhibit  progress  even  if  company  visionaries 
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understand  the  details.  In  turn,  metrics  monitoring  programs  make  project  benefits  visible  and 
hold  executive  interest. 

Without  documented  processes,  automation  is  not  possible.  In  fact,  automation  can  be 
detrimental  in  those  circumstances.  As  the  defense  industry  consolidates,  companies  acquire 
various  design  processes  and  tools.  A  concerted  effort  is  required  to  drive  related  segments  of 
the  business  to  standard  tools  and  processes.  Having  done  that,  information  flow  and  process 
automation  can  be  addressed. 

Designers  and  business  managers  are  faced  with  multiple  data  sources  and  heterogeneous 
design  information.  Typically,  only  a  few  data  experts  exist  who  know  how  to  find  all  of  the 
information  applicable  to  a  given  problem.  Data  sources  must  be  integrated  in  a  manner  that 
makes  information  readily  available  to  the  user.  The  fewer  human  interfaces  required  in  the  data 
flow,  the  fewer  the  errors  that  will  be  introduced.  The  bottom  line  is  that  businesses  must  define 
data  requirements  and  dataflow  in  much  the  same  as  processes. 

Discipline  and  metrics  are  the  final  keys  to  be  addressed.  As  with  physics,  business  tends 
towards  disorganization  unless  forcefully  directed  otherwise.  Design  process  documentation  and 
tool  standardization  are  of  little  benefit  if  they  are  not  used.  Design  reviews  and  process  reviews 
are  requirements  of  a  successful  business.  Similarly,  those  reviews  must  present  viable  metrics 
that  can  be  monitored  for  further  process  enhancement.  None  of  this  happens  of  its  own  accord. 
Disciplined  design  only  occurs  when  it  is  required  by  corporate,  program,  and  functional 
management. 

3.0  METRICS 

Baseline  Legacy  System  and  Compare  to  Pilot.  The  CPOM  Metrics  task  shows  that  the 
use  of  COTS  parts  together  with  implementation  of  best  practices,  polices  and  procedures  and 
new  DMS  management  tools  has  had  both  a  positive  and  negative  impact  on  cost  avoidance  in 
the  design,  manufacturing  and  sustainment  of  military  electronic  hardware.  However,  we 
believe  the  data  shows  more  of  a  positive  than  negative  impact. 

Material  cost  has  decreased  50%  while  design  complexity  has  doubled  on  average.  This 
has  allowed  more  capability  in  less  space  at  less  cost.  The  majority  of  COTS  devices  are 
available  in  packaging  ready  for  automated  assembly  as  compared  to  military  where 
considerable  manual  intervention  was  required.  Thus,  we  are  seeing  an  average  40%  decrease 
in  manufacturing  cost  with  a  10-20%  decrease  in  build  time. 

The  use  of  COTS  parts  in  new  designs  is  not  without  its  problems.  We  are  experiencing 
higher  cost  in  the  handling  of  plastic  COTS  parts  because  of  moisture  sensitivity  issues. 

Moisture  exposure  is  closely  monitored  and  controlled.  In  addition,  many  COTS  devices  are 
only  available  in  gold  plated  leads  and  are  unsuitable  for  surface  mount  technology  (SMT). 

Gold  leaded  parts  introduce  gold  embrittlement  reliability  issues  and  are  banned  from  Northrop 
Grumman  SMT  manufacturing.  Gold  leaded  parts  must  be  removed  from  their  packaging,  hot 
solder  dipped,  and  repackage  before  production  use.  We  are  also  seeing  a  300%  increase  in  the 
number  of  DMS  occurrences  as  a  result  of  COTS  usage,  indicating  much  shorter  life  cycles  as 
commercial  industry  continues  to  drive  the  market.  However,  cheaper  COTS  part  prices  has 
lowered  “life  of  type  buy  cost”  and  lengthened  design  refresh  intervals. 
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While  COTS  parts  are  not  without  their  faults,  there  is  considerable  cost  avoidance  to  be 
realized  from  their  use.  However,  there  is  a  greater  need  for  better  DMS  management  practices 
as  life  cycles  shortened  and  the  need  for  more  precise  design  refresh  planning  grows. 


Table  3-1  Design  Metrics 


Design  Metrics 

Metric 

%  Change 
Digital 

%  Change 

RF 

Comments 

1 

Number  of  jumpers 

100%  decrease 

700%  increase 

3  tol  increase  in  RF  design  complexity  level 

2 

Number  of  design  revisions 

40%  decrease 

33%  increase 

3  tol  increase  in  RF  design  complexity  level 

3 

Number  of  electrical  board  components 

35%  increase 

25%  decrease 

RF  devices  more  integrated 

4 

Number  of  standard  (COTS)  parts 

N/A 

N/A 

5 

Number  of  custom  parts 

N/A 

N/A 

6 

Per  cent  of  standard  (COTS)  parts 

50%  increase 

40%  increase 

7 

Per  cent  of  custom  parts 

94%  decrease 

82%  decrease 

8 

Number  of  active  parts 

N/A 

N/A 

9 

Number  of  passive  parts 

N/A 

N/A 

10 

Per  cent  of  active  parts 

8%  decrease 

47%  decrease 

RF  devices  more  integrated 

11 

Per  cent  of  passive  parts 

5%  increase 

24  %  increase 

12 

Redesign  time 

No  change 

No  change 

13 

Redesign  complexity 

N/A 

N/A 

14 

Change  in  electrical  part  count 

N/A 

N/A 

15 

Build  time 

20%  decrease 

10%  decrease 

16 

Test  time 

7%  decrease 

15%  increase 

3  tol  increase  in  RF  design  complexity  level 

17 

First  time  factory  yield 

10%  increase 

33%  increase 

18 

Factory  yield 

10%  increase 

18%  increase 

19 

Number  of  DMS  last  time  buys  made 

N/A 

300%  increase 

COTS  life  cycles  are  shorter 

20 

Number  of  DMS  part  replacements 

N/A 

100%  increase 

COTS  life  cycles  are  shorter 

21 

Number  of  new  sources  developed 

N/A 

N/A 

22 

Number  of  additional  redesigns 

N/A 

N/A 

23 

Age  of  design 

N/A 

N/A 
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Table  3-2  Financial  and  Sustainment  Metrics 


Financial  Metrics 

Metric 

%  Change 
Digital 

%  Change 

RF 

Comments 

1 

Material  cost 

50%  decrease 

60%  decrease 

2 

Design  cost  (NRE) 

35%  decrease 

45%  decrease 

3 

Production  cost 

55%  decrease 

35%  decrease 

4 

Test  cost 

5%  decrease 

20%  increase 

3  tol  increase  in  RF  design  complexity  level 

5 

Repair  cost 

26%  decrease 

38%  increase 

3  tol  increase  in  RF  design  complexity  level 

6 

DMS  last  time  buy  cost 

N/A 

100%  increase 

COTS  life  cycles  are  shorter 

7 

Sell  Price 

47%  decrease 

32%  decrease 

Sustainment  Metrics 

Metric 

%  Change 
Digital 

%  Change 

RF 

Comments 

1 

Predicted  reliability  (LRU) 

66%  increase 

280%  increase 

2 

Demonstrated  reliability  (LRU) 

290  %  increase 

N/A 

3 

Number  of  failures 

N/A 

N/A 

4 

Number  of  repairs 

N/A 

N/A 

5 

Number  of  DMS  parts 

N/A 

300%  increase 

COTS  life  cycles  are  shorter 

6 

Per  cent  of  DMS  parts 

N/A 

300%  increase 

COTS  life  cycles  are  shorter 

7 

Planned  redesigns  vs  unplanned 

N/A 

N/A 

8 

Predicted  component  obsolescence 

N/A 

N/A 

9 

Predicted  board/module/SRU  obsolescence 

N/A 

N/A 

3.1  Cost  avoidance 

•  40%  -  50%  increase  in  the  use  of  COTS  parts 

•  80%  -  90%  decrease  in  the  use  of  custom  parts 

•  10%  -  20%  decrease  in  build  time 

•  10%  -  30%  increase  in  first  time  factory  yield 

•  300%  increase  in  DMS  occurrences 

•  200%  average  increase  in  reliability 

3.2  Cost  Savings 

•  50%  -  60%  decrease  in  material  cost 

•  35%  -  45%  decrease  in  redesign  cost  (NRE) 

•  35%  -  55%  decrease  in  production  cost 

•  30%  -  40%  decrease  in  sell  price 
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4.0  CONCLUSIONS 


CPOM  was  extraordinarily  successful  in  documenting  greater  than  22  million  dollars  in 
annual  cost  reductions  from  processes  and  tools.  These  reductions  have  been  reflected  in 
Northrop  Grumman  quoting  procedures  thereby  benefiting  all  new  proposals. 

Key  CPOM  activities  contributing  to  the  cost  savings  were:  enhanced  reliability 
modeling,  Commodity  Design  Teams,  Common  Product  Teams,  improved  thermal  design, 
improved  material  handling  and  improved  manufacturability.  These  enhancements  have  been 
incorporated  into  design  processes  to  enable  COTS  technologies 

Key  CPOM  tool  activities  included:  Poet  -  process  and  tool  integration,  metrics,  MOCA 
-  obsolescence  planning  and  cost  minimization,  RADSS  -  obsolescence  management  and 
decisions,  12  -  CDT/CPT  workflows,  DMS  data,  parametric  searching,  Rosetta  -  design 
capture,  process  modeling  and  VectorGen  -  test  vector  generation.  These  tools  effectively 
address  real-world  concerns.  Several  have  been  incorporated  into  standard  Northrop  Grumman 
processes  and  a  couple  of  more  have  stimulated  interest  for  intrasector  tool  deployment. 

CPOM  metrics  gathering  demonstrates  program  impact  on: 

>  23  major  product  developments 

>  1 1  Antennas 

>  19  Transmitters 

>  24  Processors 

>10  Low  Power  RF 

>  3  Inertial  Measurement  Technologies 

>  22  Power  Supplies 

>  200  Digital  and  Microwave  Boards 

Throughout  its  nearly  five-year  span,  CPOM  has  been  of  great  benefit  to  Northrop 
Grumman  and  to  our  customers.  The  processes  and  tools  developed  here  have  resulted  in  lower 
cost,  more  reliable,  and  more  sustainable  products.  Northrop  Grumman  has  appreciated  the 
opportunity  to  work  with  AFRL  ManTech  and  would  be  pleased  to  carry  on  these  initiatives  in 
the  future. 
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